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Com puter Tuto r^llan d Execution Inter face 
Operable over Network Connection 





Technical Field 

The present invention relates generally to computers, and, specifically, to an 
execution system operable to perform tutorial operations, expert system operations, 
document preparation operations, financial analysis preparation, and various other functions 
in conjunction with various computer application programs and also operate various 
peripheral components to perform operations without requiring a specific interface to each 
application program and/or peripheral component. 



Background of the Invention 

15 For many years, application programs, on-line services, and other computer 

application software have been available for use with digital computers. Application 
programs performed word processing functions, numeric functions, data-base functions, 
accounting functions, inventory control functions, and a wide variety of other functions. 
Application programs served not only to increase the efficiency of their user but to increase 

20 the user's accuracy as well. On-line services allowed a user to access large databases of 
information that could be downloaded and used as desired. 

Application programs may be used in document preparation, financial analysis 
preparation, communication preparation, monitoring of the operation of systems and 
components through connected peripheral devices, and numerous other functions. 

25 Application programs, however, generally operate in isolation, not having the ability to 
interface with other programs, except those of a common vendor, at best. When application 
programs do work together, it is typically through the common formatting and interchange 
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of data. Thus, no content is typically provided by the interaction of application programs. 

Users of the application programs operate in isolated environments with the programs. 

When first operating an application program or on-line service, a significant amount 

of time is required to educate the user in the use of the computer application software or 
5 information services. Not until the user is sufficiently trained in the use of the computer 

application software or service may substantial benefits be derived from the program or 

service. 

The earliest approaches to training users were made by the providers of computer 
application software. The first training tools involved written instruction books that were 

1 0 included with the programs. These books described the functions available in the software, 
how to implement the functions, and the limitations of the functions. The earliest training 
books were written in a highly technical manner that prevented the average user from 
gaining a thorough understanding of the program. Resultantly, a large industry grew around 
providing written training materials for training users to use the programs. Over time, with 

15 increased competition, the written materials became easier to read and understand, providing 
more thoughtful approaches to educating the user. 

While the software industry developed, some persons and organizations recognized 
the shortcomings of the written book type instruction and stepped in to provide classroom 
and interpersonal instruction. Classroom type instruction targeted specific software that was 

20 popular enough to justify the large capital expenses associated with this type of training. 
Levels of instruction varied from lectures given in large auditoriums all the way down to 
one-on-one training sessions. While this type of instruction proved to be quite successful 
due to its human aspect, it was very expensive and generally required that the new user leave 
his or her place of employment to attend. Further, because the user generally was not 

25 provided with hands-on training, and even if he or she was, the training was not performed 



WO 98/22865 PCT/US96/18477 
on his or her own machine. Because of differences in machines and environments used in 
the training classes, the user could not always transfer the knowledge he or she had obtained 
to his or her own computer. 

Over time, vendors of the software and others in the industry recognized the value 
5 of training the user on his or her own machine while inside the software itself Thus, on-line 
tutorials were developed. On-line tutorials typically combined a written description of a 
particular function of the software and instruction in specific commands that would allow 
a user to perform the function. These on-line tutorials typically allowed a user to perform 
a few instructions at a time as directed by the tutorial with the instructions being monitored 

10 to ensure correctness of operation. While these on-line tutorials provided the benefit of 
learning while doing, they were typically difficult to follow and did not provide adequate 
explanation. Part of the problem related to inadequate written explanation that carried over 
from the user's manual was that such information was conveyed to the user only in a written 
format displayed on the screen. Further, because they were specific to the particular 

15 application program, they did not provide a familiar reference frame for the user, and the 
user first had to learn to use the on-line tutorial in the particular program. 

Thus, attempts were made to combine the benefits of classroom training with the 
benefits of hands-on training on the user's own machine. A few vendors recorded classroom 
training programs on video cassettes so that a user could play the topics at his or her own 

20 speed on a nearby television while simultaneously working on the computer. Thus, a user 
could combine the benefits of working on his or her own machine while also obtaining the 
benefits of being in a classroom. Unfortunately, there was no interplay between the video 
being viewed and the user's commands issued to the computer application software. While 
this system allowed the user to play the video in his of her office, it did not provide the 

25 interactive benefits available from other techniques. Thus, the system did not reinforce the 
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commands described in the video and required the simultaneous operation of two separate 
machines. 

One visual teaching aid, sold under the tradename LOTUS SCREENCAM, displays 
images on a computer screen that are identical to those displayed within an application 
5 program. However, even though the teaching aid displays images that a user would 
encounter during use of the program, the teaching aid merely functions like a video player. 
The teaching aid merely displays to the user a proper sequence of keystrokes and/or mouse 
movements that would be required to execute specific functions and does not provide 
interaction between the user and the actual application program. 

10 Even when a user becomes proficient in the use of an application program, the user 

must provide ali content to documents produced by the application program. Thus, the 
training performed and expertise gained typically has no relationship to particular job tasks 
to be performed. Some application programs included wizard applications or functions that 
aided the user in performing certain tasks within the application programs. These wizard 

1 5 applications are a form of internal control or operation of the application programs. But the 
variety of tasks capable of being executed by such internal controls is typically limited by 
several factors, including the fact that the variety of tasks is buried into the internal structure 
of the application itself and thus not readily accessible, and the internal level of complexity 
added by these internal controls or operations prevent vendors from covering a broad range 

20 of capabilities. 

Another problem faced by currently-available systems relates to the control of 
peripheral devices connected to computer systems. These devices may comprise interface 
circuitry allowing the computer system to function with equipment, such as process 
controllers, manufacturing equipment, and other equipment that may be computer 

25 controlled. Software for controlling these devices was typically highly specific to the type 
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of equipment being controlled, could not interact with other computer programs, and 
provided little ability for users to vary the operation of connected equipment. 

Thus, there lies a need for a multipurpose execution system that interfaces between 
application programs, local data bases, remote data bases, expert systems, and peripheral 
5 devices to perform various functions including training, control, and assistance to a user. 



Summary Of The Invention 

The invention herein is provided to overcome the above-described limitations, and 
others, of the prior tutorials, application program interfaces and execution systems. The 

10 present invention, therefore, includes a user interface, a data retrieval interface, a data 
interpreter, a target application interface, and a peripheral device interface. The user 
interface transmits an instruction set to a user and receives instructions from the user based 
upon the instruction set. One of the received instructions selects a target application or 
peripheral device from a plurality of independently executable applications or peripheral 

1 5 interfaces. 

The data retrieval interface retrieves data from storage based upon user input while 
the data interpreter receives the data and the instructions from the user and, based upon the 
target application and the data, transforms the data into actions. The target application 
interface receives at least some of the actions and selectively issues some of the actions to 

20 operate the target application. Finally, the peripheral interface receives at least some of the 
actions and selectively issues some of the actions to operate the peripheral device. Thus, the 
system of the present invention provides a multi-purpose generalized interface to any 
application program or peripheral device. 

The present invention retrieves data and instruction sets from either a local data 

25 archive such as a CD ROM or magnetic disk or retrieves instructions and data from a remote 
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storage location over a data link. The instructions include generalized task sets and 
transformation components. The transformation components allow the system to transform 
the generalized task sets for particular application programs or peripheral devices. Thus, the 
system of the present invention has a wide range of applicability. 
5 More particularly, one embodiment of the system provides a computer-based tutorial 

interface that provides a user with audio/visual training on specific Sanctions of a computer 
program or computer services, provides actual samples of the implementation of the 
functions, assists the user in learning to perform the function within the computer application 
software, and requires the user to take an active approach in the training by performing 
10 actual instructions within the application software on the user's own machine. 

Moreover, other aspects of the present invention will become apparent with further 
reference to the drawings and specification which follow. 



Brief Description Of The Drawing 

15 FIG. 1 is a block diagram illustrating a computer system that may be utilized in 

conjunction with the present invention; 

FIG. 2 is a fimctional block diagram illustrating the functional relationship between 
elements of an embodiment of a system embodying the principles of the present invention; 
FIG. 3 illustrates a sample display screen of an interface of a system or the present 
20 invention performing tutorial functions detailing a tutorial topic for a paint-type application 
program; 

FIGs. 4a and 4b illustrate executions accomplished by the system of FIGs. 2 and 3 
relating to particular selections made by a user of available functions; 
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FIG. 5 illustrates a further embodiment of the invention with an action flow of 
instructions and data among the elements of the system wherein the instructions are 
selectively interpreted and forwarded based upon a selected target application; 

FIG. 6 is a block diagram of the system of the present invention illustrating the flow 
5 of instructions and feedback; 

FIG. 7 is a flow diagram illustrating the various action steps involved in retrieving 
data from either a local or a remote storage location; 

FIB. 8 is a flow diagram illustrating the various action steps involved in issuing 
instructions to the appropriate interface, that is, a target application or a peripheral device 
1 0 interface; 

FIG. 9 is illustrates a conventional format of an instruction file including initialization 
instruction header and the sequence of elements of an elementary instructions; and 

FIG. 10 is an example of a library field structure that can be available to the system. 

1 5 Detailed Description of a Preferred Embodiment 

In the following description, certain details are set forth to provide a complete 
understanding of the present invention. It will be apparent to one skilled in the art, however, 
that these specific details are not required in order to practice the present invention. Also, 
well known electrical structures and circuits are depicted in block diagram form so as not 
20 to obscure the present invention unnecessarily. 

The descriptions which follow are presented in part in terms of operations within a 
computer. These descriptions and representations are the means used by those skilled in the 
software arts to most effectively convey the substance of their work to others skilled in the 
art. 
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An algorithm is here, and, generally, conceived to be a self-consistent sequence of 
steps leading to a desired result. These steps are those requiring physical manipulations of 
physical quantities. Usually, though not necessarily, these quantities take the form of 
electrical or magnetic signals capable of being stored, transferred, combined, compared, and 
5 otherwise manipulated. It proves convenient at times, principally for reasons of common 
usage, to refer to these signals as, values, symbols, characters, display data, terms, numbers, 
or the like. It should be borne in mind, however, that all of these and similar terms are to 
be associated with the appropriate physical quantities and are merely used here as convenient 
labels applied to these quantities. 

10 Further, the manipulations performed are often referred to in terms, such as 

comparing, commonly associated with mental operations performed by a human operator. 
No such capability of a human operator is necessary, or desirable in most cases, in any of the 
operations described herein which form part of the present invention, since the operations 
are machine operations. Useful machines for performing the operations of the present 

15 invention include general purpose digital computers or other similar devices. The present 
invention relates to method steps and apparatus for operating a computer in processing 
electrical or other physical signals to generate other desired physical signals. 

The present invention also relates to a system for performing these operations. This 
system may be specifically constructed for the required purposes, or it may comprise a 

20 general purpose computer as selectively activated or reconfigured by a computer program 
stored in the computer. The algorithms presented herein are not inherently related to any 
particular computer or other apparatus. In particular, various general purpose machines may 
be used with programs written in accordance with the teachings herein, or it may prove more 
convenient to construct more specialized apparatus to perform the required method steps. 
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The required structure for a variety of these machines will appear from the description 
below. 

In the following description, several terms are used frequently, have specialized 
meanings in the present context, and are thus defined. The terms "environment," 
5 "windowing environment," and "running in windows" are used interchangeably to denote 
a computer user interface in which information is manipulated and displayed within bounded 
regions on a raster scanned video display. 

The terms "application," "computer application software," and "program" are used 
interchangeably herein to refer to any computer program run in conjunction with the present 
1 0 inventive system. Such computer programs could relate to computer applications, with on- 
line services, communication systems, or any other computer oriented function. 

The term "current" is sometimes used herein as an antecedent to "window," 
"application," etc., and is used to denote system components which are currently being 
utilized or performing operations with respect to a particular computer application software 
1 5 running in the environment. 

A system 10 embodying the principles of the present invention is shown by way of 
illustration in FIGS. 1-4. The system 10 of the present invention is implemented on a typical 
computer system 1 1 as shown in FIG. 1 . This computer system 1 1 typically comprises a 
CPU 12, a computer bus 14, a disc drive 16, main memory 1 8, a compact disc drive 20 f and 
20 user interface components. These user interface components preferably comprise a mouse 
22 and mouse controller 24, a video display 26 and video display controller 28, and a 
keyboard 30 and keyboard controller 32. Preferably, the computer system 1 1 also includes 
an audio interface 34 that transmits audio information to and receives audio information 
from a user of the system. 
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As one skilled in the art will appreciate, the system and method of the present 
invention are implemented on the computer system 1 1 but are not readily identifiable as 
specific components of the system. Those skilled in the art will readily understand how the 
described invention may be implemented on any of a variety of computer systems. 
5 Therefore, the implementation of the system on a particular hardware platform will not be 
more fully described herein. 

Referring now to FIGS. 2 through 4, a computerized, multimedia tutorial interface 
system 10 for training a user to use computer application software comprises control display 
means 40, instruction input means 42, instruction interpretation means 44, audiovisual 
10 enablement means 46, computer application software interface means 48, and computer 
application software control means 50. 

Referring specifically to FIGS. 2 and 3, the control display means 40 comprises a 
control bar 51 and a topic selection screen 53 T each of which are selectively displayed on the 
computer screen 26. Together, the control bar 5 1 and the topic selection screen 53 provide 
15 a plurality of instructions to a user that may be selected by the user. As is shown, the 
control bar 5 1 preferably has commands similar to those found on a video tape player, 
including exit, rewind, goto, fast forward, stop, back, pause, and play. The topic selection 
screen 53 allows a user to access video segments relating to specific topics to be learned or 
specific operations to be executed. Typically, the user accesses specific video segments on 
20 a topic-by-topic basis as desired. In the preferred embodiment, the topic selection screen 
53 is displayed only when certain commands are executed via the control bar 5 1 . However, 
depending upon the application, the topic selection screen 53 could also be continuously 
displayed. 

The instruction input means 42 operates to receive an instruction from a user 49. 
25 Preferably, the instruction input means 42 combines hardware and software components. 

-10- 
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In the preferred embodiment of the present invention, the instruction input means 42 
comprises the combination of the mouse 22, the keyboard 30, the control bar 51, the topic 
selection screen 53, and related software that allows the user 49 to select a desired function. 
Selecting a command from a menu displayed on a computer screen 26 using a mouse 22 and 
5 via a keyboard 30 are both well known in the art and are not fully described herein. As one 
skilled in the art will readily appreciate, however, the instruction input means 42 could also 
include the combination of the audio interface 34 in conjunction with voice recognition 
software. 

The instruction interpretation means 44 interprets the user instruction, creates at least 

10 one execution instruction, and selectively issues the execution instruction. The instruction 
interpretation means 44 preferably comprises a series of software instructions executed on 
the computer system 1 1 in a fashion well known in the art. For example, when the user 
selects an instruction via the instruction input means 42, software code monitors the mouse 
controller 24 and keyboard controller 32 interfaces, receives input from the interfaces, 

15 processes the input to determine what function has been executed, and issues the proper 
execution instruction to the respective system component. 

Still referring to FIGS. 2 and 3, the audiovisual enablement means 46 operates to 
receive execution instructions from the instruction input means 44, to selectively retrieve 
audiovisual information responsive to the execution instruction, and to display the 

20 audiovisual information on the computer screen 26. Preferably, the audiovisual information 
comprises a video clip that is retrieved from a compact disc via the CD drive 20 or the data 
link. The video clip is then decoded, formatted, and displayed on the computer screen 26 
in a video window 55. The video window 55 may cover only a portion of the computer 
screen 26 or may be expanded to be as large as the screen. Preferably, as is shown in FIG. 

25 3, the video window 55, the control bar 51, and the topic selection screen 53 all reside on 



WO 98/22865 PCT7US96/18477 
top of the computer application software window 57 when they are active. However, when 
they are inactive, they are all hidden. Further, in the preferred embodiment, the topic 
selection screen 53 is displayed only when certain user instructions are executed. 

In a typical use of the tutorial interface system, the user 49 selects a specific video 
5 clip that corresponds to a particular topic and topic to be learned. The video clip is then 
retrieved and displayed on the computer screen 26 in the video window 55. After the 
information has been displayed, and, if the process has not been aborted or otherwise 
interrupted by the user, control may be returned to the user or be given to another system 
10 component. In the preferred embodiment, immediately after the video clip has been 

10 displayed, or during a user 49 initiated break in the video clip, control is taken again by the 
computer application software interface means 48. However, the system 10 may also be 
operated such that the computer application software interface means 48 takes control 
during a video clip, halts the video clip to demonstrate a function or service feature, and then 
later restarts the video clip. 

15 The computer application software interface means 48 also receives execution 

instructions from the instruction interpretation means 44. The computer application 
software interface means 48 interfaces directly with computer application software and 
selectively executes a function of the computer application software that is described in a 
video clip. Thus, the functions performed by the computer application software interface 

20 means 48 within the computer application software provides a second visual training tool 
to the user 49 on the computer display 26. Preferably, the function or set of functions 
executed within the computer application software relate directly to the audiovisual segment 
that was just displayed to the user 49. Preferably, the computer application software 
interface means 48 comprises a communication agent that is loaded into main memory at 

25 system 10 startup. - The communication agent accesses instruction sets specific to the 

-12- 
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computer application software of interest that are stored in separate files on the disc drive 
16. Thus, to perform a specific set of instructions within the computer application software, 
the communication agent brings the computer application software up on the computer 
display 26, accesses the instructions, and then executes the instructions within the computer 
5 application program. 

The computer application software control means 50 selectively relinquishes control 
of the computer application software to the user 49 so that the user may practice operating 
the computer application software. The computer application software control means 50 
also selectively regains control of the computer application software from the user 59. In 
10 this fashion, the user may practice those techniques that were previously described to him 
via the video clip and also were performed by the computer application software interface 
means 48. As one skilled in the art will readily appreciate, the computer application 
software control means 50 is preferably implemented as a combination of software 
instructions. 

15 Preferably, the system 10 of the present invention also comprises user instruction 

monitoring means 52, error message issuance means 54, and evaluation means 56, all of 
which provide feedback to the user when the user has control of the computer application 
software. Specifically, the instruction monitoring means 52 monitors the user instructions 
issued to the computer application software, keeping track of the instructions. When 

20 activated, the error message issuance means 54 issues an error message to the user on the 
computer screen 26 if the user issues instructions that are erroneous. To determine whether 
the issued instructions are erroneous, the error message issuance means 54 compares the 
user's issued instructions to a list of correct instructions. Further, when activated, the 
evaluation means 56 evaluates the instructions issued to the computer application software 

25 by the user and issues a summary of the user's performance in issuing the instructions. 
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Thereby, the evaluation means 56 provides an indication of the user's performance in 
learning to use the computer application software. As one skilled in the art will readily 
appreciate, the user instruction monitoring means 52, error message issuance means 54, and 
evaluation means 56 are all preferably implemented as a combination of software instructions 
5 and executed accordingly. 

Referring specifically to FIG. 4, the operation of the tutorial interface system 10 is 
described. In the description of the system 10 operation, each relevant system event is 
identified with a numeral in parentheses. Immediately after the system 10 is started (100), 
the interface with the computer application software is initiated, and the control bar 5 1 and 

1 0 video window 55 are created (102). Next, the interface between the main program and the 
communication agent is established (104), and the communication agent is loaded into main 
memory 18. At this point, the product logo is displayed, and an introduction video segment 
may be played (108) on the computer screen 26. The system 10 then prompts the user to 
enter an instruction from the control bar 5 1 . Immediately upon entering the program, a 

15 topic index is set to a predetermined value and a topic index is also set to a predetermined 
value. When the program is run for the first time, these two indexes are set at one. 
However, when the user 49 continues with a previously-started topic, the indexes may be 
automatically set to those of the prior session. 

Each instruction available on the control bar 5 1 may be executed by the user 49. The 

20 EXIT instruction (110) provides notification of an exit to the communication agent via the 
communication agent (112), stops the communication agent (114), and closes all 
communication agent command files (116). The EXIT instruction (110) optionally further 
shuts down the core program (118), closes the control bar 51 window, the video window 
55 (120), and ends the current execution or training session (122). 



.14- 
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Executing the REWIND (RWD) instruction (124) with a double click causes the 
current topic index and the current topic index, as displayed in the topic selection screen 
window 53, to index to the first topic and first topic (126). Executing the REWIND (RWD) 
instruction with a single click causes the current topic index to decrement by a single topic 
5 (126). 

Executing the GOTO instruction (128) opens the topic selection screen 53 and 
allows the user to select a particular topic to be indexed (130). Then the user 49 has the 
option of playing the video clip or the demonstration of the particular topic. Depending 
upon the option selected, the video clip plays or the execution plays (130). 
10 Executing the FAST FORWARD (FF) instruction (132) with a double click causes 

the current topic index and the current topic index, as displayed in the topic selection screen 
window 53, to index to the last topic (134). Executing the FAST FORWARD (FF) 
instruction (132) with a single click causes the current topic index to increment by a single 
topic (134). 

15 Executing the STOP instruction (136) causes the process that is running when the 

instruction is executed to stop immediately (138). Executing the BACK instruction (140) 
freezes the current topic indexes and plays the previously-viewed video clip again (142). 
The PAUSE instruction (144) causes the currently-playing video clip, if one is playing, to 
stop for later continuation (146). 

20 The PLAY instruction (148) first causes the topic counter to increment (1 50). The 

topic counter indexes the relevant video clip and communication agent instructions. The 
next video clip is then played in the video window 55 (152). Then, the control bar 5 1 and 
the video window 55 are hidden, and the communication agent instructions may be executed 
(154). A notification of play is transmitted to the communication agent (156), the order is 

25 received by the communication agent (158), and instructions are read from the 
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communication agent library and executed within the computer application software (160). 

Once the instructions are completed, a backwards notification is sent (162) so that the 

control bar 51 and the video window 55 are again displayed (164). Next, a short, written 

synopsis of the demonstration that was executed is displayed to the user 49 on the computer 
5 screen 26 (168). Then, the computer application software interface means 48 allows the 

user to practice within the computer application software to enforce what he or she has 

learned (168). Then, the user's performance is evaluated (170). 

The COUNTER INFO instruction (172) toggles the time information displayed 

between elapsed time from the start of the video clip to the time remaining in the video clip 
10 (1 74). Optionally, the counter could also display the time since the user 49 logged on or the 

clock-time. The counter hide instruction (176) toggles the counter display between being 

hidden or displayed (178). 

The system 1 0 of the present invention can be easily implemented with application 

programs, on-line services, or any computer application software. The system 10 is generic 
1 5 and provides a familiar training interface that can be used in many varied situations. As one 

skilled in the art will readily appreciate, the system 1 0 of the present invention is readily 

transportable to provide tutorial instruction in any computer based system. 

Description of a Further Embodiment 
20 A further aspect of the present invention has an engine component for accessing local 

or remote instruction file libraries and for controlling peripheral devices. 

Referring to FIG. 5, a general action flow 500 of the system 10 is shown. The action 
flow has two portions, the first being the engine interaction portion 501 and the results area 
503. 



-16- 



WO 98/22865 PCT/US96/18477 

The engine interaction portion 501 has four actions: a user instruction action 502 
where selections are received from the user, an instruction input action 504 where 
instructions are retrieved, and an instruction interpreter action 508 where the instructions 
are executed or translated into use. Application selector action 506 is to ensure coverage 
5 of the aspect of the engine 501 being a "juke box" for instruction files, so that the engine can 
externally operate a wide variety of target applications. The term "externally" means that 
the control or operation activity comes from without the target application. In contrast, for 
example, an internal control or operation is commonly provided by a function wizard such 
as those that accompany Microsoft® Windows or Excel. 
10 Upon start-up of the action flow, the user instruction action block (502) is provided 

by the control bar 51 (shown in Fig. 3). At user instruction action block (502), the choices 
of either a target application (506) or a tutorial program is selected from topic selection 
screen 53 (shown in Fig. 3) because an instruction program typically has a plurality of 
instruction files that can be executed randomly. That is, one of the possible user instructions 
1 5 is the choice of which "clip" or inserted file to play through the control bar 5 1 (shown in Fig. 
3). Other available user instructions are user preferences for running the program such as 
speed and screen display options. Afterwards, at instruction input action (504), engine 702 
fetches instructions that are remotely or locally located. Afterwards, after the fetched 
instructions are loaded into memory, the engine interprets the instructions accordingly at the 
20 instruction interpreter action block (508). 

The choice of application is also a form of user instruction action (502). For 
example, if application programs are on the computer, such as Microsoft® Works or Excel, 
then the engine 702 presents a variety of instructions, one of which is to start the application 
program. That is, user 49 selects the target application, being Microsoft® Word for 
25 example, for which may exist a dedicated set of instructions, which are disassociated from 
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the user instructions 502. The instruction interpreter 508 is then called, which chooses one 
instruction file having a set of instruction related to Microsoft® Word and then returns along 
flow path 51 1 to the user instructions action 502 to receive the user's next instruction. 

When an instruction set is ready, the next user instruction, which is typically "Play," 
5 but can also be "rewind," or other instructions as set out in the discussion regarding Fig. 3 
earlier herein. The action flow then retrieves (in the instruction input action (504)) and 
translates the instruction into an action at the instruction interpreter action 508. The engine 
continues to "play" until the end of the instruction file 1000 is reached (see Fig. 12), 
discussed later herein. 

10 Upon interpretation of the instruction files at the instruction interpreter action 508, 

the result area 503 is entered through the instruction interpreter action (508). The result area 
503 has an audiovisual interface action (510), and application interface action (512), and an 
application control action (514). That is, the appropriate media data is played accordingly. 
For example, if a video clip was selected, then the clip is decoded, formatted, and displayed 

1 5 on the computer screen 26 in video window 55 (see Fig. 3). The audiovisual interface action 
(510) plays multimedia information. The application interface action (512) is the interface 
of application program functions with the computer peripherals such as mouse, display, 
keyboard and the like. The application control action (514) is the result of putting the 
application peripherals into action. In the case of application control action (514), checking 

20 whether everything is operating and execution is suspended or stopped accordingly in the 
target application. That is, the system polls the target application for status checks regarding 
problems or the like. For example, when computer error occurs, an error message is 
broadcast. So control action (514) polls the program so that the user is informed 
accordingly. 
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Referring to Fig. 6, a detail of system 10 is shown with an engine 702 having 
interfaces for instruction/data storage 700 7 a user 704, target application 708, a peripheral 
device 710, and a host 712. It should be noted that other interfaces can be had. 

Engine 702 has a plurality of software components with designated software 
5 functions. Engine 702 has an interpreter component 716, and an interpreter component 716 
that routes instructions to the appropriate application or peripheral device. Interpreter 716 
is in bi-directional communication with an instruction/data interface component 714, a target 
interface component 720, and peripheral interface component 722. Interpreter component 
716 is in uni-directional communication with user interface 718. 
10 Target interface 720 and user interface 718 are components of system 10 shown in 

Fig.s 1-4. The capabilities added to the engine 702 as shown in Fig. 6 are the 
instruction/data interface 714 and peripheral interface 722 components adding new and 
additional device capabilities to the system. 

The instruction/data interface component 714 can be constituted of either a local 
15 location or remote network interface location. Location instruction/data interface 
component 714 is a set of functions that load or access files either locally or remotely. The 
term "locally" means a physical data storage device such as CD drive 20 or disk drive 16, 
shown in Fig. 1 . The term "remotely" means a data link, established either through a local 
area network ("LAN 7 ') or a wide area network ("WAN") or the Internet as set out under 
20 conventional network standard protocols. A suitable network standard is TCP/IP 
("Transport Control Protocol/Interface Program")- 

Referring to Fig. 8, an instruction/data storage component program flow chart is 
shown. After startup (801), the engine screen is displayed (see Fig. 3). Engine 702 also 
determines the system hardware configuration (802). After the configuration is established, 
25 the engine 702 prompts the user 49 for a choice of which instruction to execute (804). 
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Upon an input selection by user 49, engine 702 determines whether data files are local (806). 
If files are local (808), then engine 702 retrieves the designated files (820). If the data files 
are not local (808), then engine 702 establishes communications network connection 810. 
The communications network connection can be established over a conventional computer 
5 communications devices. Examples of suitable devices are a conventional modem or ISDN 
("Integrated Services Digital Network") communications links using conventional software 
communications services. Upon establishment of the connection (810), the user's identity 
is verified (812), After verification, the required instruction files are downloaded to the 
engine 702 and either stored in local memory or executed on-the-fly. Preferably, in light of 

10 the current technology, the files are stored locally. But as technology permits, other 
execution schemes can be implemented with equal effectiveness. 

At this point, the retrieve files instruction step (820) and the download required files 
step (814) converge. The next step for the engine 702 is to execute the initialization 
instructions included in the files that were retrieved either locally or remotely (816). 

15 Referring back to Fig. 6, the interpreter reads the initialization instructions obtained in step 
(816) and routes accordingly. That is, the interpreter prepares the operating environment 
of the target application or the target application itself for the instruction files to be 
effectively executed by the engine 702. For example, if the retrieved instruction files require 
that the main window of the target application occupy all the screen, the engine 702 will 

20 have the target application occupy all the screen. Referring back to Fig. 7, the user is 
prompted for input (818). The input being the options provided by the control bar 51 (Fig. 
3) such as "play," "rewind" or the like. 

Referring back to Fig. 6, the input provides directions for execution on either the 
target interface component 720 or the peripheral interface 722 depending on the instruction 

25 type. The peripheral interface 722 is the capability by which the engine 702 can issue 
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instructions specific to particular peripheral devices 710 responsive to external-computer 
signal control such as computer printers, industrial robotics, robotic drones or the like. 
Conventional peripheral devices 710 typically have their own user-interface apart from the 
engine 702 interface component 718. 
5 Referring to Fig. 8, the program waits for user input (822). Upon the user input, the 

engine 702 identifies the instructions necessary in response to the user 49 input from the 
control bar 51 (826). Next, the engine 702 determines the application target of the user's 
instruction (828). Based upon the target determination (702), the engine will know if the 
instructions require translation in that, if the instruction is for a target application (830), the 

10 instruction will be executed directly (834). If the instruction is for a peripheral device (830), 
then the instruction will be translated (832) and changed in its contents in order for it to be 
issued to the peripheral device in the device's communication medium (832). Then the 
instruction is issued and executed to the appropriate application 708 or device 710 (834). 
Feedback is issued from the application 708 or device 710 (834) to the user 49 

15 through the screen 26 shown in Fig 1 (828) The user at any point has the ability to 
terminate the process. If the user 49 does not elect to terminate the process, the process will 
continue until the end of the retrieved instruction file is reached and that all instructions have 
been executed or an error has occurred that led the engine 702 to stop execution by itself 
Referring to Fig. 9, shown is an instruction file 1000 format having a sequence filed 

20 for each command. For example, each instruction in an instruction file typically has four 
parameters: "time stamp," "type of instruction," "parameter 1," and "parameter 2." The 
fields are not separated, and each instruction is not separated from the previous or the 
following ones. On each instruction file, there is typically a header block containing 
parameters relating to the environment needed for the instruction about to be executed. For 

25 example, if a target application needs to occupy all the screen 26, the header block dictates 
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that the sequence of instructions requires this environment. This header block is executed 
in the initialization instructions in step 816 of Fig. 8. 

Referring to Fig. 10, an example of a library field structure that can be available to 
the system 10 is shown. An advantage to the system 10, which implements the engine 702 
5 (shown in Fig. 7), is that remote access to industrial machinery in semiconductor clean 
rooms or in hazardous areas can be readily had with equipment implementing computer 
communications interfaces, for example, the RS-232-C standard for serial communications 
interfaces or the V.32 recommendation standard for 9600-bps modems. System link 910 is 
the link between instruction/data storage 700 and instruction/data interface 714. That is, 
10 system link 910 is the hardware connection to libraries 902 through 914, which can be 
located remotely on the Internet, LAN or, WAN systems. Referring to the clean-room 
environment, the engine 702 can access instruction files in these libraries that have processes 
for controlling semiconductor fabrication equipment remotely. Otherwise, when such 
equipment fails, the plant must be closed down while engineers, specially outfitted in 
15 particle-free suits, repair the clean-room equipment. A loss in production and profit occurs 
during the fabrication plant shut-down. The universal capability of using a general personal 
computer to remotely operate industrial equipment is greatly desired. 

The above-described preferred embodiments are intended to illustrate the principles 
of the invention, but not to limit the scope of the invention. Various other embodiments and 
20 modifications to these preferred embodiments may be made by those skilled in the art 
without departing from the scope of the following claims. 

In view of the above-detailed description of the present invention and associated 
drawings, other modifications and variations will now become apparent to those skilled in 
the art. It should also be apparent that such other modifications and variations may be 
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effected without departing from the spirit and scope of the present invention as set forth in 
the claims which follow. 
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1 . A computer-executable system that selectively operates any of a plurality of 
independently-executable applications, the system comprising: 

a user interface that transmits an instruction set having a plurality of 
instructions for selection by a user and receives at least one selected instruction based upon 
said instruction set, said at least one selected instruction designating a target application 
from a plurality of independently-executable computer applications; 

a data retrieval interface that retrieves a plurality of data from a computer 
memory, said plurality of data based upon said at least one selected instruction; 

a data interpreter that receives said data and said at least one selected 
instruction, said data interpreter translates said data into a plurality of actions with respect 
to said target application and said data; and 

a target application interface that receives at least some of said plurality of 
actions and that selectively issues some of said plurality of actions for externally operating 
the target application and that relays feedback from the target application back through the 
data interpreter to the user interface. 

2. The system of claim 1, wherein said data additionally includes a video clip 
for the target application that the data interpreter passes to the user interface for display to 
the user. 
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3. The system of claim 2, wherein after externally operating the target 
application, the data interpreter selectively passes control of the target application to the user 
through the user interface. 

4. The system of claim 3, wherein the data interpreter further comprises: user 
instruction means, error message issuance means, and evaluation means, whereby tutorial 
feedback is provided to the user when the user has control of the target application. 

5. The system of claim 1, further comprising: 

a peripheral interface that receives at least some of said plurality of actions 
and selectively issues some of the actions for operating a peripheral device having an 
embedded instruction set and a communications port and that relays feedback from the 
peripheral device back through the data interpreter to the user interface, thereby 
demonstrating the operation of the peripheral device to the user. 

6. The system of claim 5, wherein said data additionally includes a video clip 
for the peripheral device that the data interpreter passes to the user interface for display to 
the user. 

7. The system of claim 5, wherein after externally operating the peripheral 
device, the data interpreter selectively passes control of the peripheral device to the user 
through the user interface. 
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8. The system of claim 7, wherein the data interpreter further comprises: user 
instruction means, error message issuance means, and evaluation means, whereby tutorial 
feedback is provided to the user when the user has control of the peripheral device. 

9. The system of claim 1, further comprising: 

a host interface that receives at least some of said plurality of actions and that 
selectively issues some of the actions for directly operating a host system. 

10. The system of claim 1, wherein the data retrieval interface comprises: 

a peripheral storage device controller for interfacing with a local storage 
device storing said data. 

11. The system of claim 1, wherein the data retrieval interface comprises: 

a network interface for accessing an other device over a network and for 
retrieving said data stored on said other device. 

12. The system of claim 1, wherein the data retrieval interface comprises: 

a wireless interface for accessing an other device over a wireless link and for 
retrieving said data stored on said other device. 
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13. A system that selectively operates wy^M$\^3dlity±o( ^MifelctS98 
independently-operable peripheral devices, the system comprising: 

a user interface that transmits an instruction set having a plurality of 
instructions for selection by a user and that received at least one selected instruction based 
upon said instruction set, said at least one selected instruction designating a target peripheral 
device from a plurality of independently-operable peripheral devices 

a data retrieval interface that retrieves a plurality of data from a computer 
memory, said plurality of data based upon said at least one selected instruction; 

a data interpreter that receives said plurality of data and said at least one 
selected instruction, said data interpreter translates said data into a plurality of actions with 
respect to aid target peripheral device and said plurality of data; and 

a peripheral interface that receives at least some of said plurality of actions 
and selectively issues some of said actions for externally operating a peripheral device having 
an embedded instruction set and a communications port for interfacing with said peripheral 
interface and that relays feedback from the peripheral device back through the data 
interpreter to the user interface. 



14, The system of claim 13, further comprising: 

a target application interface that receives at least some of said plurality of 
actions and selectively issues some of said plurality of actions for operating a target 
application selected from a plurality of target applications and that relays feedback from the 
target application back through the data interpreter to the user interface. 
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15. The system of claim 13, wherein said data additionally includes a video clip 
for the target application that the data interpreter passes to the user interface for display to 
the user 



16. The system of claim 15, wherein after externally operating the target 
application, the data interpreter selectively passes control of the target application to the user 
through the user interface. 



1 7. The system of claim 1 6, wherein the data interpreter further comprises: user 
instruction means, error message issuance means, and evaluation means, whereby tutorial 
feedback is provided to the user when the user has control of the target application. 



18. The system of claim 13, further comprising: 

a host interface that receives at least some of said plurality of actions and 
selectively issues some of said plurality of actions for directing an operation of a host system. 



19. The system of claim 13, wherein the data retrieval interface comprises: 

a peripheral storage device controller that interfaces with a local storage 
device for accessing said plurality of data. 

20. The system of claim 13, wherein the data retrieval interface comprises: 

a network interface for accessing an other device containing said plurality of 
data over a network. 
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21 . The system of claim 14, wherein the data retrieval interface comprises: 

a wireless interface for accessing an other device containing said plurality of 



data over a wireless link. 



22. A computer-implemented method for selectively operating any of a plurality 
of independently-executable computer applications, the method comprising the steps of: 

receiving from a user a selected instruction from a set of instructions, the 
selected instruction designating a target application from the plurality of executable 

computer applications; 

retrieving from a computer memory a plurality of data with respect to the 

selected instruction; 

translating the plurality of data into a plurality of actions with respect to the 

target application; 

transmitting the plurality of actions to the target application for externally 
operating the target application; and 

relaying feedback from the target application back to the user. 

23 . The method of claim 22, wherein the plurality of data additionally includes 
a video clip related to the target application, and further comprising the step of displaying 
the video clip to the user. 

24. The method of claim 23, wherein after externally operating the target 
application, selectively passing control of the target application to the user. 
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25. The method of claim 24, further comprising the step of: providing tutorial 
feedback to the user when the user has control of the target application. 
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26. A computerized, multimedia tutorial interface system for training a user to 
use any of a plurality of independently executable computer application software programs, 

the system comprising: 

(a) computer application program selector, wherein the computer application 
5 software program selector receives an instruction from a user and causes the 

computerized, multimedia tutorial interface system to select a computer 
application software program from the plurality of independently executable 
computer application software programs; 

(b) control display means for displaying a control window on a computer screen, 
1 o the control window providing a plurality of instructions to a user that may 

be selected by the user, wherein the instructions include instructions for 
displaying audiovisual images describing operations of the selected computer 
application software program; 

(c) instruction input means for receiving a user instruction from a user; 
1 5 (d) instruction interpretation means for receiving the user instruction from the 

instruction input means, for interpreting the user instruction based upon the 
selected computer application software program to generate an execution 
instruction, and for selectively issuing the execution instruction to element 
(e), (£) or (g) based upon the user instruction; 
20 (e) audiovisual enablement means for receiving an execution instruction from the 

instruction interpretation means, for selectively retrieving audiovisual 
information responsive to the execution instruction, and for displaying the 
audiovisual information on at least a portion of a computer screen, wherein 
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the audiovisual information includes audiovisual images describing 
operations of the selected computer application software program; 

(f) computer application software interface means for receiving an execution 
instruction from the instruction interpretation means, for interfacing with the 
selected computer application software program, and for issuing simulated 
user input to the selected computer application program to execute a 
function of the selected computer application software program that is 
described in the audiovisual image; and 

(g) computer application software control means for receiving an execution 
instruction from the instruction interpretation means, for causing the 
computerized, multimedia tutorial interface system to completely relinquish 
control to the computer application software program to allow a user to 
execute functions within the computer application software program, and for 
causing the computerized, multimedia tutorial interface system to gain 
control from the computer application software program based upon user 
input. 



27. The computerized, multimedia training system of claim 25 further 
comprising: 

(a) user instruction monitoring means for monitoring user instructions 
issued to the instruction input means by a user. 
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28. The computerized, multimedia training system of claim 25 further 
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comprising: 



(a) error message issuance means for issuing an error message to a user 
on a computer screen if the user issues a user instruction to the 
instruction input means that is erroneous as compared to expected 

instructions. 



29. The computerized, multimedia training system of claim 25 further 
comprising: 

(a) evaluation means for evaluating the user instructions issued to the 
instruction input means by a user and for issuing a summary of the 
user's performance in issuing the user instructions. 



30. The computerized, multimedia training system of claim 25 wherein the 
control display means comprises a display bar having commands selected from a group of 
commands including at least exit, rewind, goto, fast forward, stop, back, pause, and play. 

31. The computerized, multimedia training system of claim 25 wherein the 
audiovisual information is displayed in a window on a computer screen adjacent to, and not 
overlying, the control window. 
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32. A computerized, multimedia tutorial interface method for training a user to 
use any of a plurality of independently executable computer application software programs, 
the method comprising the steps of: 

(a) displaying a list of the plurality of independently executable computer 
application software programs to a user, 

(b) receiving a software application program selection instruction from 
the user and, based upon the software application program selection 
instruction, selecting an application software program from the 
plurality of independently executable computer application software 
programs; 

(c) displaying a control window on a computer screen, the control 
window providing a plurality of instructions to a user that may be 
selected by the user, wherein the instructions include instructions for 
displaying audiovisual images describing operations of the selected 
computer application software program; 

(d) receiving a user instruction from a user; 

(e) based upon the selected computer application software program, 
interpreting the user instruction to generate an execution instruction; 

(f) based on the execution instruction and the selected computer 
application software program, selectively retrieving audiovisual 
information; 

(g) based on the execution instruction, displaying the audiovisual 
information on at least a portion of a computer screen, wherein the 
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audiovisual information includes audiovisual images describing 
operations of the selected computer application software program; 
based on the execution instruction, executing a function of the 
computer application software program that is described in the 
audiovisual information by issuing simulated user input to the 
selected computer application software program; 
based on the execution instruction, completely relinquishing control 
to the selected computer application software program to allow a 
user to execute functions within the selected computer application 
software program; and 

based on user input, regaining control from the selected computer 
application software program. 

33. The computerized, multimedia training method of claim 32 further 
comprising the step of: 

(a) monitoring user instructions issued by a user. 

34. The computerized, multimedia training method of claim 33 further 

comprising the step of: 

(a) issuing an error message to a user on a computer screen if a user 
issues a user instruction that is erroneous as compared to reference 
user instructions. 
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35. The computerized, multimedia training method of claim 32 further 

comprising the steps of: 

(a) evaluating the user instructions issued by a user, and 

(b) based upon the user instructions issued by the user, issuing a 
summary of the user's performance in issuing the user instructions. 

36. The computerized, multimedia training method of claim 32 wherein the step 
of displaying a control window on a computer screen includes displaying on a control bar 
commands selected from a group of commands including at least exit, rewind, goto, fast 
forward, stop, back, pause, and play. 

37. The computerized, multimedia training method of claim 32 wherein in the 
step of displaying the audiovisual information on at least a portion of a computer screen, the 
audiovisual information is displayed in a window on a computer screen adjacent to, and not 
overlying, the control window. 
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IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 

In re Application of: Edward S. Mann, II Attorney Docket No. : 1 73 22PCTUS 

and Frederick Milliot 

For; COMPUTER TUTORIAL AND EXECUTION INTERFACE 
OPERABLE OVER NETWORK CONNECTION 



DECLARATION FOR UTILITY PATENT APPLICATION 

Hon. Commissioner of Patents 

and Trademarks 
Box Applications 
Washington, D.C. 20231 

Sir; 

As the below named inventors, we hereby declare that; 

Our residences, post offices addresses and citizenships are as stated below next to our 

names. 

We believe we are the original, first, and joint inventors of the subject matter which is 
claimed and for which a patent is sought on the invention entitled COMPUTER TUTORIAL AND 
EXECUTION INTERFACE OPERABLE OVER NETWORK CONNECTION, as described 
in PCT International Patent Application Serial No. PCT/US96/18477, filed 18 November 1996 
(18.11.96). 

We hereby state that we have reviewed and understand the contents of the above-identified 
specification, including the claims, as amended. 

We acknowledge the duty to disclose information which is material to the examination of 
this application in accordance with Title 37, Code of Federal Regulations, § 1.56. 

We hereby claim foreign priority benefits under Title 35, United States Code, § 1 1 9(a)-(d) 
or § 365(b) of any foreign application(s) for patent or inventor's certificate, or § 365(a) of any PCT 
international application which designated at least one country other than the United States o f America, 
listed below and have also identified below any foreign application for patent or inventor's certificate 
or of any PCT international application having a filing date before that of the application on which 
priority is claimed and as amended. 
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Prior Foreign Application 

Number(s) Country 



Foreign Filing Date 
(MM/DD/TYYY) 



Priority 
Not Claimed 



Certified Copy Attached 
Yes No 



PCT/US96/18477 



PCT 



11/18/1996 



None 



X 



We hereby claim the benefit under Title 35, United States Code § 1 1 9(e) of any United States 
provisional application(s) listed below. 

Application Numbers) Filing Date (MM/DD/YYYY) 
None 

We hereby claim the benefit under Title 35, United States Code § 120 of any United States 
application(s), or § 365(c) of any PCT international application designating the United States of 
America, listed below and, insofar as the subject matter of each of the claims of this application is not 
disclosed in the prior United States or PCT international application in the manner provided by the first 
paragraph of Title 35, United States Code, § 1 12, we acknowledge the duty to disclose information 
which is material to patentability as defined in Title 37, Code of Federal Regulations, § 1.56 which 
became available between the filing date of the prior application and the national or PCT international 
filing date of this application. 

US Patent Application PCT Parent Parent Filing Date Parent Patent Number 

Number Number (MM/DD/YYYY) (if applicable) 

08/541,221 None 10/12/1995 5,577,186 

We hereby declare that all statements made herein of our own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable by fine 
or imprisonment, or both, under § 1001 of Title 18 of the United States Code and that such willful 
statements may jeopardize the validity of the application or any patent issued thereon.. 
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Inventor's signature: 




Edward S. Mann, II 




Citizenship: U.S.A. 

Address : 915 Zaragoza Street 
Laredo, Texas 78040 



Date 



Inventor's signature: 



Frederick Milliot 



Citizenship: France 



Address: 44, rue du Vieux Pont 

91800 Boussy St-Antoine 
France 



Date: 



vb\c \gmny\ mann.5VdecLus 




DECLARATION FOR UTILITY PATENT APPLICATION - Page 3 of 3 



17322PCTUS 



Inventor's signature: 



Date: 



Edward S. Mann, II 

Citizenship: U.S.A. 

Address: 9 1 5 Zaragoza Street 
Laredo, Texas 78040 



Inventor's signature: 








Frederick Milliot ^ 


< 


Citi^nship: France 





Date: 



Address: 44, rue du Vieux Pont 

91800 Boussy St-Antoin e 
France 



vb\c.\gmny\ mann.5NdecLus 
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